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(54) Title: AESTHETIC IMAGING SYSTEM 
(57) Abstract 

Disclosed is an aesthetic imaging system (20) for use in editing digital images. The aesthetic 
imaging system includes an imaging program (21) that runs on a personal computer (28) having an 
image capture board (30), a monitor (32), a video source (34) for providing digital images to be 
edited by the aesthetic imaging system, and a pen and tablet (38) for use in editing the images. The 
imaging program includes a unique combination draw tool that includes a freehand draw mode, a 
curve mode and an undo mode that are available without cycling through menus. The combination 
draw tool may be used with any of the draw tools. Another feature of the imaging program is 
autoblend, a rectangular user interface that is invoked by each of the shape tools. The autoblend 
interface simplifies editing when using shape tools by consolidating the move, paste and blend, and 
paste without blending commands into a single, convenient interface. 
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AESTHETIC IMAGING SYSTEM 
Relationship to Other Applications 
This application is a continuation-in-part of U.S. patent application Serial 
No. 08/406,201, filed March 17, 1995, and U.S. patent application Serial 
5 No. 08/617,439, filed March 18, 1996, the benefit of the filing of which is hereby 
claimed under 35 U.S.C. § 120. 

Field of the Invention 
This invention generally relates to computer imaging programs and, more 
specifically, to a method and apparatus for manipulating digital photographs. 
10 Background of the Invention 

The digital age continues to present additional opportunities for visual 
communication using computers. As an example, digital photographs are routinely 
being manipulated to produce a desired effect or result in the magazine and film- 
making industries. In the medical field, computer-based imaging has and continues to 
15 gain acceptance in a clinical setting as a viable communications tool between plastic or 
"cosmetic" surgeons and potential patients. 

People are with increasing frequency consulting physicians about cosmetic 
surgery. While in many cases the patients considering cosmetic surgery have an 
impressive understanding of the procedures available and medical terms used to 
20 describe these procedures, it is apparent that the slightest miscommunication may 
result in dire consequences. This has promoted the use of computer imaging to 
facilitate communication between the physician and prospective patient. Specifically, 
high-end aesthetic imaging systems allow a physician to take pre-operative digital 
images of the patient, e.g., including profile and frontal views. The images are stored 
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physician's changes. It is therefore difficult for the physician to show various 
combinations of the edits that had been performed. For example, a physician may edit 
an image to remove wrinkles around a patient's eyes and to narrow the patient's nose. 
Existing aesthetic imaging programs only allowed the physician to simultaneously 
S show all of these changes. I£ for example, the patient wished to view the changes to 
the eyes without the changes to the nose, it was difficult for the physician to easily 
separate out the sequence of edits that had been performed to adjust the patient's 
nose. An improved aesthetic imaging system in which a physician can more easily edit 
pre-operative images in response to a patient's suggestions and inquiries would be 

10 extremely advantageous. 

A still further disadvantage of existing aesthetic imaging systems is that the 
systems allow a physician to perform nearly flawless editing of a patient's image. 
Unfortunately, the edits performed by a physician on an aesthetic imaging system are 
often unobtainable results that cannot be achieved when actual surgery is performed. 

IS Unless the physician is especially skilled at using the aesthetic imaging system, it is 
difficult to show the patient achievable results, which typically fall within a range 
somewhere between the original patient image, and the optimum results as displayed 
by the edited image on the screen. It therefore would be advantageous to develop an 
aesthetic imaging system that allowed a physician to display more realistic results that 

20 are achievable through surgery. 

Summary of the Invention 
The invention is an aesthetic imaging system for use in editing digital images. 
The aesthetic imaging system includes a unique user interface that allows edits to be 
performed more efficiently and with less confusion to the patient. 

25 In one aspect of the invention, a method of editing a digital image comprised 

of a plurality of color pixels in an aesthetic imaging system is disclosed. The aesthetic 
imaging system including a processor, a memory, a monitor, and a pen and 
cooperating tablet for controlling a cursor displayed on the monitor. The pen has a 
depressable tip and a side button, each of which include an on status and an off status, 

30 wherein the position of the pen tip relative to the tablet determines the position of the 
cursor on the monitor. The method comprises: (a) evaluating the following variables: 
(i) the status of the tip of the pen; (ii) the status of the side button on the pen; 
and (iii) movement of the pen tip relative to the tablet; (b) actuating a freehand 
drawing mode if a first set of variables are present, wherein movement of the pen 

35 relative to the tablet edits pixels that are located at positions corresponding to the 
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menu bar. Only the commands and the outline of the menu bar are presented in a 
contrasting color, minimizing the overall visual impression created by the menu bar 

In yet another aspect of the invention, a warp tool is described that allows a 
user to quickly and easily manipulate various features in an image. To use the warp 
tool, a user first encircles a portion of the image to be edited. Once the area has been 
selected, the user may tip the pen to designate a stretch point within the selected area 
As the user floats the pen over the tablet, the image is then stretched as if pulled from 
the stretch point. Areas of the image in the direction of stretch are compressed, and 
areas away from the direction of stretch are expanded. Areas surrounding the 
warpmg area are automatically adjusted to ensure that there is no discontinuities with 
the warpmg area. The manipulation of the image is performed in real-time, allowing a 
patient to see the warping as it is being performed by a user. 

In accordance with still another aspect of the invention, a number of modules 
are provided to allow the user to improve the quality of an image, to analyze an 
image, or to prepare an image for meetings and presentations. A color correction 
module allows the color of an original image to be closely matched with the color of a 
target image. An orientation correction module allows the size and orientation of an 
original image to be closely matched with the size and orientation of a target image 
A measurement module allows angles, distances, areas, and proportions to be 
measured and recorded on an image for presentation to a patient or colleagues And a 
label module allows structures in a patient's image to be linked to textual descriptions 
The modules greatly improve the ability to compare two images or maintain accurate 
records of achievable surgical results. 

In accordance with yet another aspect of the invention, a module is also 
provided that simulates the effect of laser resurfacing treatment on a patient The 
module accurately portrays the image of the patient immediately following surgery 
the image of the patient during the healing process, and the image of the patient when" 
the patient has fully healed. 

An advantage of the tools, features, and modules described herein are that they 
improve the overall experience of a patient during a preoperative visit with a 
physician. The powerful tools in the aesthetic imaging system allow the physician to 
easily manipulate the patient's image in response to feedback provided by the patient 
The aesthetic imaging system interface also allows the patient to focus on the image 
being manipulated, rather than on the aspects of the aesthetic imaging system that 
allow the manipulation. The end result is an improved preoperative visit that provides 
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FIGURE 1 1 is a pictorial representation of a user interface for implementing 
the autoblend tool of FIGURE 10; 

FIGURE 12A is a flow chart of an exemplary routine illustrating a cutout tool 
in accordance with the invention; 
5 FIGURE 12B is a flow diagram of an exemplary routine illustrating a rotate 

tool in accordance with the invention; 

FIGURE 13 is a flow chart of an exemplary routine for viewing images in 
accordance with the invention; 

FIGURES 14A-14D are pictorial representations illustrating the effects of a 
10 compare feature in accordance with the invention; 

FIGURES 15A-15C illustrate a split image option of viewing images in 
accordance with the invention; 

FIGURE 16 is a pictorial representation illustrating the use of a translucent 
image to allow a patient to accurately position themselves in order to capture a second 
15 image having the same location and orientation as an original stored image; 

FIGURE 17 is a pictorial representation illustrating a compare image wherein 
a presurgical image of a patient is compared side-by-side with a postsurgical image 
having the same location and orientation; 

FIGURES 18A-18C are pictorial representations illustrating the use of a warp 
20 shape tool to edit a patient's image; 

FIGURES 19A-19E are pictorial representations illustrating the function of 
the warp tool; 

FIGURE 20 is a pictorial representation illustrating the use of a transparent 
menu bar when viewing an image of a patient; 
25 FIGURE 21 is a flow chart of an exemplary routine for implementing a zoom 

viewing feature in accordance with the invention; 

FIGURES 22A-22D are flow charts of exemplary routines for implementing a 
number of modules that may be used to optimize an image for presentations or 
display; 

30 FIGURE 23 is a pictorial representation illustrating the use of a color 

correction module in accordance with the invention; 

FIGURE 24 is a pictorial representation illustrating the use of an orientation 
correction module in accordance with the invention; 

FIGURES 25A-25D are pictorial representations illustrating the use of a 
35 measurement module in accordance with the invention; 
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Indianapolis, Indiana. The buffers are discussed in regard to a single pose only, such 
as a profile or front view of a person. 

The original image buffer 50 contains an unedited digital image, for example, a 
side profile picture of a potential patient. The modified image buffer 52 contains any 
5 edits made to a copy of the original image. The modified image buffer is updated 
during a save and after each session. The current image buffer 54 contains 
information identical to the modified image buffer upon beginning a session. 
Thereafter, edits made to the current image are saved in the working buffer 56 as an 
overlay to the current image. During a save, the contents of the current image 
10 buffer 54 are copied to the modified image buffer 52, and the working buffer 56 is 
cleared. 

Prior to discussing the aesthetic imaging system in further detail, a 
compendium of terms used in the application may be helpful: 

1 5 Image A digital photograph or picture of a patient. 

Stylus The "pen" that may be used to select menus, modify 

images, and carry out other commands in the 
program. The stylus controls the cursor, just as a 
mouse pointing device does on a personal computer. 



20 



25 



Tablet, or 

Pad The electronic notepad used in conjunction with a 

stylus. The pen must be held relatively close to the 
pad in order for the pen to communicate with the 
tablet. Unlike a mouse, the tablet follows an X/Y 
grid that matches the monitor, i.e., if the pen is 
positioned at the top left comer of the tablet, the 
cursor is displayed at the top left corner on the 
30 monitor. 



Floating 



Moving the pen to move the cursor, without actually 
touching the tablet. 
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wants to load an image (picture) from computer storage, e.g., a hard drive. If the 
Storage option was selected, the image(s) to be edited or viewed during the current 
session are selected at block 70. 

At block 72, the selected images are copied to the appropriate buffers in the 
5 frame grabbing board, as described in FIGURE 2 and the accompanying text. For 
example, if the selected image is an original image that has not yet been edited, the 
original image will be copied to the original, modified, and current image buffers. If 
the selected image is an image that has previously been modified, the original image is 
copied to the original image buffer SO and the modified image is copied to both the 

10 modified and current image buffers 52 and 54. It will be appreciated that the number 
of images that may be loaded at one time will be limited, in part, by the capacity of the 
frame grabbing board. The program then loops to block 62. 

If the Storage option was not selected, a test is made at block 74 to determine 
if the Draw option has been selected from the main menu. If the Draw option has 

15 been selected, a draw tool routine is called at block 76. The program then loops to 
block 62. A suitable routine for implementing the Draw option is shown in 
FIGURE 5. 

If the Draw option was not selected, a test is made at block 78 to determine if 
the Shape option has been selected. If the Shape option has been selected, a shape 
20 tool routine is invoked at block 82. A suitable routine for implementing the Shape 
option is shown in FIGURE 10. Otherwise, at block 82 a test is made to determine if 
the View option has been selected. 

If the View option has been selected, a view subroutine is invoked at block 83. 
An appropriate routine for the View option is shown in FIGURE 13. The program 
25 then loops to block 62. 

If the View option was not selected, a test is made of block 84 to determine if 
the Modules option has been selected. If the Modules option has been selected, a list 
of the modules available to the user is provided at block 85. Suitable routines for 
implementing the various modules are shown in FIGURES 22A-22D. If the Modules 
30 option is not invoked, the routine continues to a decision block 86. 

At block 86 a test is made to determine whether the Exit option has been 
selected from the main menu. If not, the program loops to block 62. Otherwise, any 
edits to the image are saved at block 88. At this point in the program, the image in 
the current image buffer 54 is saved to the modified image buffer 52, and the working 
35 buffer 56 is cleared. The program then terminates. 
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monitor. Patients can thus easily align themselves with their former picture to achieve 
very similar before and after pictures. 

The advantage of the translucent method of aligning a patient is shown in the 
side-by-s.de display shown in FIGURE 17. As shown in FIGURE 17, the aesthetic 
unagmg system of the present invention may generate a side-by-side display of the 
two images to allow a patient to easily and accurately compare a presurgical picture 
w,th a post-surgical picture. The left half of the monitor may display the presurgical 
.mage 370, and the right half of the monitor may display the post-surgical image 372 
Allowmg a patient to view the two images side-by-side in precisely the same 
onentauon provides the patient with an accurate impression of the results achieved bv 
surgery. 3 

Upon establishing a desired position for the patient, the image is focused and 
s.zed at block 102 by using the aesthetic imaging system to adjust the electronic 
controls on the video camera. After any adjustments have been made to the camera, 
at block 104 the tip of the pen is pressed anywhere on the tablet to freeze the digital 
•mage onto the monitor. At block 106, the user makes a determination if the image 
currently displayed on the monitor is acceptable. If the image is not acceptable the 
routine loops to block 100. If the image is acceptable, an appropriate command is 
entered at block 108 and the image is stored in nonvolatile memory for future 
20 viewing. 

At block 1 10, a test is made to determine if an exit or other similar command 
has been entered by the user, i.e., if any more pictures are to be taken. If additional 
p.ctures are to be taken, the program loops to block 100. Otherwise at block 1 1 1 a 
checksum value (described below) is calculated by the imaging program for each 
(ongmal) unage that has been stored. At block 1 12, the imaging program stores each 
.mage and its associated checksum value. The routine then returns to block 68 of 
FIGURE 3. 

tt Determining Authenticity T Tcinp Checksum V a »,„.c 

The checksum value is an addendum to an original stored image that is used to 
determine its authenticity when the image is subsequently displayed or printed Those 
sidled in the art will recognize that there are a number of methods of implementing 
such a checksum procedure. For example, one checksum computation is to add up 
the grayscale values for one of the colors, i.e., red, green, or blue, for each pixel 
compnsmg the .mage. Assuming a screen of 640 by 480 pixels and 256 colors per 
p«el, the checksum values would range from 0 to (640X480)(255). When an image 
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original image is acquired and when an image is to be displayed, or the comparison 
will be meaningless. 

While the use of a checksum is contemplated in the exemplary routine for 
determining whether an original image has been altered, it will be appreciated that 
5 other techniques may be used to detect when an original digital image has been 
modified. For example, a flag or other marker may be uniquely associated with the 
original digital image. If the flag or other marker is absent from the digital image 
being displayed, the digital image is a copy that is presumed to have been changed. 
Alternatively, two versions of the image may be stored, included an unaltered original 

10 and a copy. The two images may be compared in order to determine whether 
modifications have been made to the copy. 

The editing aspects of the invention are now described. 
m. Editing Using Draw Tools 

A disadvantage of prior art aesthetic imaging systems is that a physician or 

15 facilitator may have to cycle back and forth between several menus in order to 
properly edit an image. In an effort to minimize the number of menus required, the 
imaging program includes a unique combination draw (CD) feature that generally 
works with all of the drawing tools. The CD feature allows a user to freehand draw, 
use curves to edit an image, as well as undo using either freehand or curves, without 

20 having to invoke a separate menu for each item 

The following describes an interface for the CD feature as implemented in an 
embodiment of the invention. For purposes of this discussion, it will be assumed that 
the airbrush tool is selected as the drawing tool, although it is to be understood that 
the CD tool generally applies to all of the drawing tools. Upon selection of the 

25 airbrush draw tool, the aesthetic imaging system prompts the user to choose a color 
from a color palette that appears on the monitor. A color is selected using the pen. 
After selecting a color, a side bar menu is displayed. The user may select from a 
number of options on the side bar, including brush size and intensity, or select away 
from the side bar menu to remove the menu from the screen. In the latter case, the 

30 system defaults are used. 

To freehand draw, the user presses on the tablet with the pen tip and continues 
pressure while moving or "rubbing" the pen on the tablet. At this point the chosen 
color is written onto the image at the location on the monitor that corresponds to the 
pen location. Pressing the side bar while repeating the motion will allow the user to 

35 selectively remove any edits to the image using a freehand motion. 
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was pressed against the tablet, the imaging program enters freehand draw mode, 
shown at block 130. In one embodiment of the invention, freehand draw mode is 
entered if the pen moves the equivalent of three or more pixels. While in freehand 
draw mode, freehand edits may be made to the image in a manner similar to prior art 
5 imaging programs. This mode will remain until pressure on the pen tip is released. 
After beginning freehand draw mode, the routine loops to block 124. 

If the pen was not moved, e.g., the pen has been moved two pixels or less 
prior to releasing the tip, the imaging program begins a curve mode by establishing a 
first endpoint, as indicated at block 132, and drawing a line on the monitor from the 

10 endpoint to the current pen position, indicated at block 134. At block 136, a test is 
made to determine if the tip has been pressed. The imaging program at this point is 
looking for a second endpoint to be entered. If not, the program loops back to 
block 136 to await the input. As noted above, the user can go back to the beginning 
of the routine using the cancel button if the user has changed his or her mind, although 

15 this is not shown in the flow diagram. Specifically, the first cancel would place the 
routine at block 124, the second at either block 122 or 124, depending on the draw 
tool selected, and subsequent cancels would forward the routine to the main menu. 

Once the tip is pressed, the second endpoint is established at the tip position, 
and a line segment is drawn on the monitor from the two endpoints, as indicated at 

20 block 138. At this point in the routine, the imaging program is in "curve draw mode" 
as indicated at block 140, and the user can make any edits desired using a curvilinear 
line segment having the established endpoints. The routine loops to block 124 while 
in this mode. 

With reference to block 124, if the side button has been depressed a test is 
25 made at block 142 to determine if the imaging program is in either the freehand or 
curve draw modes. If the imaging program is in either mode, an undo mode will be 
invoked as long as the side button remains depressed, as indicated at block 144. If at 
the time of depressing the side button the imaging program is in freehand draw mode, 
the undo will also be freehand. Selective freehand undo edits are thus available. 
30 Similarly, if at the time of depressing the side button the imaging program is in curve 
draw mode, the undo will be in this mode. Selective "undo" edits are then available 
using a curve, as opposed to freehand drawing. Releasing the side button will return 
the imaging program to the drawing mode that was active just prior to depressing the 
side button. 
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Step S4 
Step SS 
Step S6 
Step S7 
Step S8 

Step S9 

Step S10 

Step Sll 

Step S12 

FIGURE 7B: Step S13 

Step S14 
Step S15 
Step S16 
Step S17 
Step S18 
Step S19 
Step S20 
Step S21 

FIGURE 7C: Step S22 

Step S23 
Step S24 
Step S25 
Step S26 
Step S27 
Step S28 
Step S29 
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select airbrush; 

move pen to curve; 

select curve; 

select an airbrush color; 

move pen to the first anchor point 

position; 

select at the position to establish the 
anchor point 136; 

move pen to the second anchor point 
position; 

select at the position to establish the 
anchor point 138; 

move pen to bend the curve 134 into the 
bridge of the nose; 

press the side button on the pen to exit 

to the main menu; 

move pen to draw; 

select draw; 

move pen to airbrush; 

select airbrush; 

move pen to freehand; 

select freehand; 

select a color for the airbrush tool; 

use a rubbing motion with the pen to 

make the freehand edit; 

press the side button on the pen to exit 

to the main menu; 

move pen to draw; 

select draw; 

move pen to undo; 

select undo; 

move pen to curve; 

select curve; 

use a nibbing motion with the pen to 
undo the previous edit; 
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In accordance with the invention, the steps required to perform the same edits 
using the aesthetic imaging system 20 are now described. With reference to 
FIGURE 8 A-8E, the steps required to perform the edits include: 
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FIGURE 8A: StepNl 
Step N2 
Step N3 
Step N4 
Step N5 
Step N6 

Step N7 

Step N8 

Step N9 

Step N10 

FIGURE 8B: Step Nil 



FIGURE 8C: StepN12 
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FIGURE 8D: Step Nil 
Step N12 
Step N13 
Step N14 



move pen to draw; 
select draw; 
move pen to airbrush; 
select airbrush; 

elect any color for the airbrush tool 
move pen to the first anchor point 
position; 

select at the position to establish the 
anchor point 136; 

move pen to the second anchor point 
position; 

select at the position to establish the 
anchor point 138; 

move pen to bend the curve 134 into the 
bridge of the nose; 

pressing the tip of the pen against the 
tablet and use a rubbing motion to make 
the freehand edit. 

pressing the tip of the pen and the side 
button simultaneously, and maintain 
pressure while rubbing in the area to be 
undone; 

move pen to the first anchor point 
position; 

select at the position to establish the 
anchor point 142; 

move pen to the second anchor point 
position; 

select at the position to establish the 
anchor point 144; 
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to be augmented. Edits made in block 206 are saved at block 208 by pressing and 
then releasing the tip of the pen. The program then terminates. 

FIGURES 9B-9C further describe the operation of the contour tool, by 
illustrating how pixels are replicated from one area of an image to another. The image 
5 areas described are for exemplary purposes only, and are simplified for clarity in this 
discussion. With reference to FIGURE 9B, an area 209 of an image is comprised of 
red218R, green218G, blue218B, and yellow 2 18Y areas separated by boundary 
lines 210, 212, 214, and 216. It is assumed that a pair of anchor points 218 and 219 
have been established by a user along the boundary 212, wherein the aesthetic imaging 

10 system will display a line segment 220 between the two anchor points. 

In FIGURE 9C, it is assumed that the user has moved the midsection of the 
line segment 220 to the right. In this instance, the blue area 218B has been stretched 
into the yellow area218Y. This area is bounded by the line segment 220 (now 
curved) and the boundary line 214. Also, the green area 218G has been stretched into 

15 the blue area 21 8B. This area is bounded by a curved line segment 221 and the 
boundary line 212. The red area 218R has expanded into the green area 218G; this 
area is bounded by a curved line segment 222 and the boundary line 210. 

If the opacity level is at 100 percent, the newly defined areas will be comprised 
of the color being expanded. Thus, the area bounded by segments 220 and 221 will 

20 be blue; the area bounded by segments 221 and 222 will be green; and the area 
bounded by segment 222 to the left edge of the diagram will be red. 

If the opacity level is less than 100 percent, pixels from the underlying image 
areas that are being written over by the newly defined areas will be blended into the 
newly defined areas. At an opacity of 80 percent, for example, the area bounded by 

25 segments 220 and 221 will still be primarily blue, but the portion of this area bounded 
by the segment 220 and the boundary line 214 may have a yellow tinge; and the 
portion of this area bounded by the boundary line 212 and segment 221 may have a 
green tinge. As the opacity percentage is dropped, the effects on these areas will be 
even greater. 

30 While somewhat simplistic, the illustration in FIGURES 9B and 9C describes 

the function of the contour tool and the effect on more complicated pixels patterns 
will be appreciated by those skilled in the art. The area bounded by segment 220, 
boundary line 214, segment 221, and boundary line 212 will not be affected by 
changes in the opacity setting. 
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With reference again to FIGURE 10, a test is made at block 244 to determine 
if the pen tip has been pressed at a location outside of the autoblend box. If the pen 
tip was pressed at a location outside the autoblend box, the edit area is pasted with a 
blending of the edges at block 260. Otherwise, a test is made at block 262 to 
5 determine if the tip location was within the approximate three-, six-, nine-, and 
twelve-o'clock areas of the autoblend box. A pressing of the tip within any of these 
areas results in the edited area being pasted without blending, as indicated at 
block 264. As indicated at block 266, a selection in a location in the autoblend box 
apart from the three-, six-, nine-, and twelve-o'clock areas will allow the image to be 

10 moved. In this case, the edited area will track movement of the pen as long as the tip 
remains pressed. After a move is completed, the routine loops to block 242. 

If a paste has been accomplished using blocks 260 or 264, a test is made at 
block 268 to determine if the user wishes to exit the shape routine, e.g., by pressing 
the side button. If not, the routine loops to block 232 where a new area of the image 

15 may be considered. Otherwise, the routine returns to block 82 of FIGURE 3. 

FIGURES 12A and 12B illustrate two exemplary shape tools that are available 
when using the aesthetic imaging system. With reference to FIGURE 12A, a cutout 
tool is unique in that a user can select an area of the image to be cut out, thereby 
creating a "hole" in the image, and an identical image underneath the cutout image can 

20 then be moved in all directions as h is viewed through the hole. The cutout feature is 
especially useful for profile views including chin augmentation, brow lifts, and 
maxillary and mandibular movement; and frontal views, including otoplasty, brow lift, 
lip augmentation, nasal base narrowing, and maxillary and mandibular movement. 

At block 270, the current image is copied to a working buffer. As is discussed 

25 in FIGURE 10, when the cutout subroutine is called the user has defined an area of 
the image to be edited. At block 272, a boundary is created around the designated 
area designated in block 234 of FIGURE 10. At block 274, the working area is 
displayed inside the boundary, and the current image displayed outside the boundary. 
In this manner, the image in the working buffer can be moved relative to the image in 

30 the current buffer until the desired alignment has been achieved. The program then 
returns to the routine of FIGURE 10. 

Upon returning to block 238, the edit may be frozen by selecting with the pen. 
Thereafter, the autoblend box is displayed. Selecting within the area 252 of the 
autoblend box allows the designated area to be moved. Selecting anywhere outside 

35 the autoblend box will make the edit permanent, with automatic blending. Selecting 
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dotted line 386. Once the area has been selected, a user may tip the pen to designate 
a stretch point within the selected area. As shown in FIGURE 18 A, a stretch 
point 388 has been designated near the top portion of the patient ! s right upper lip. 
Once the stretch point has been designated, the user may float the pen in the desired 
5 direction that they would like to stretch the image. The portion of the image that is 
located at the stretch point is pulled in the direction that the user floats the pen, with 
the area surrounding the stretch point either expanded or compressed. That is, the 
area in the direction that the user floats the pen is compressed, and the area away from 
the direction that the user floats the pen is expanded. The amount of expansion or 

10 compression is dictated by the distance of the area away from the stretch point, in a 
manner discussed in greater detail below. Preferably, the image is manipulated in real 
time, so that the user is presented with a seamless and continuous stretching or 
movement of the selected feature. 

The technique used to implement the warp tool is portrayed in 

15 FIGURES 19A-19E. A user first defines a manipulation area 420 to be edited by 
circling the desired feature of the patient. As shown in enlarged detail in 
FIGURE 19B, when the manipulation area has been selected, the aesthetic imaging 
system creates a pixel map of a warping area 422 that completely encompasses the 
manipulation area. For computational purposes, the warping area is preferably 

20 rectangular and approximately 33% larger in area that the manipulation area. It will 
be appreciated, however, that the warping area may be differently shaped or sized 
depending on the particular application and available system hardware. 

After defining the manipulation area, the user selects a stretch point 424 within 
the manipulation area by tipping the pen at a desired location within the area. As 

25 shown in FIGURE 19C, when the stretch point is selected, the aesthetic imaging 
system maps four rectangles 426a, 426b, 426c, and 426d in the warping area. One 
corner of each rectangle is defined by the stretch point, and the diagonally opposing 
corner is defined by a corner of the warping area. Perpendicular lines 427 drawn 
through the stretch point intersect the warping area boundary at points 428a, 428b, 

30 428c, and 428d. After the stretch point is selected, the user may float their pen to 
manipulate the selected feature, for example in the general direction indicated by 
arrow 430. 

As the user floats the pen to warp the selected feature, the rectangles in the 
warping area are distorted. In FIGURE 19D, the stretch point has been moved 
35 upwards and to the left in the warping area, to an intermediate location 432. 
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example, a stretch point 392 may be positioned on the upper portion of the left lip, 
and the pen floated in a direction generally designated by the arrow 394. As shown in 
FIGURE 18C, this generally causes the upper left portion of the lip to be made fuller 
to match the upper right portion of the lip. Again, during the warping the aesthetic 
5 imaging system automatically expands or contracts the surrounding unmanipulated 
area to ensure that there are no discontinuities between the upper left and the unedited 
portion of the face. 

The warp tool with multiple stretch points is a very powerful tool as it allows 
the user to quickly manipulate an image with a minimum use of drawing tools or 

10 piecemeal editing. Because the warp tool performs the manipulation in real-time, the 
edits are accomplished very quickly and fluidly. A user may therefore generate a 
desired image in a minimal amount of time. 

While the warp tool described above requires the user to define the 
manipulation area 420 to be edited prior to the creation of a warping area by the 

15 imaging system, it will be appreciated that the warping area can alternatively be 
automatically defined by the aesthetic imaging system. In such an alternative system, 
rather than defining a manipulation area by circling a region of the image, the user 
simply selects a stretch point 424 and floats the pen in the desired warping direction.. 
The imaging system automatically defines a rectangular warping area 422 of a 

20 predetermined size that surrounds the stretch point. As the user moves the stretch 
point, the area within the warping area is warped. Moreover, the size of the warping 
area 422 may also be dynamically adjusted by the imaging system. The farther the 
user floats the pen in a desired warping direction, the greater the imaging system sizes 
the warping area 422. A greater area of the patient's image will therefore be subject 

25 to the warping as the stretch point is pulled a farther distance. Automatically defining 
and dynamically adjusting the warping area allows the user to quickly and easily warp 
an image. 

V. Viewing an Image 

FIGURE 13 illustrates an exemplary routine for implementing the view 

30 features of the imaging program. In FIGURE 13, the solid blocks indicate user 
interface options presented to the user by the aesthetic imaging system and the dashed 
blocks represent system responses to the decisions made. The view group includes: 
Compare, Prioritize, Split Image, Mirror Image, and Restore to Original, as well as 
other options including Zoom and Emboss. At block 300, a test is made to determine 

35 if the Compare option has been selected. 
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With reference again to FIGURE 13, if the Compare option has been selected, 
a test is made at block 309 to determine if the entire image is to be compared or only 
certain portions of the image, i.e., using the Prioritize option. If less than the entire 
image is to be compared, the user is prompted to enter the area or areas that are to be 
5 compared at block 310. A user may then define one or more "priority areas" by 
freehand circling the desired area. When the priority areas or defined, or if all of the 
edits are to be reviewed during the comparison, at block 31 1 the user is prompted to 
float the pen in a vertical motion on top of the tablet to transition between the original 
and modified images, in accordance with the Compare feature discussed above. An 
illustration will clarify this point. 

In FIGURE 14B, a first priority area 312 has been defined that corresponds 
generally to the nose. Given this selection, the nose area only will transition from 
original to modified as the pen is moved, with the rest of the image being displayed 
unedited. Thus, the modifications to the chin and neck no longer are shown. In 
FIGURE 14C, a second priority area 314 has been defined that corresponds generally 
to the chin. The first priority area 3 12 has been kept. Given these selections, the nose 
and chin areas only will transition from original to modified as the pen is moved, with 
the rest of the image being displayed unedited. Thus, the modifications to the neck 
are not illustrated. 

With reference to FIGURE 14D, a third priority area 316 has been defined 
that corresponds generally to the neck, along with the former designations. Given 
these selections, all three priority areas 3 12, 3 14, and 3 16 transition with movement of 
the pen. Again, the undesignated portions of the image are displayed in an unedited 
form, even if parts of the image outside the priority areas have been edited (no edits 
are shown). Because edits have not been made, from a user's standpoint the transition 
in FIGURE 14D appears to be a comparison between the original and modified 
images. 

While preferably the user defines the priority areas on the image being edited, 
it will be appreciated that the priority areas may also be automatically defined by the 
aesthetic imaging system. A comparison may be made between an original image 
stored in a buffer and the edited image that has been modified by the user. Any areas 
containing differences between the original and the edited image may be highlighted 
by the aesthetic imaging system, and a priority area automatically defined for each 
area containing differences. Whether a priority area is defined may also depend on the 
number of differences between the original image and the edited image. 
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from zero to 100 percent transition (zero percent being the original image and 100 
percent being the edited image), by floating the pen up or down above the tablet to 
establish the view, and the pressing the tip of the pen against the tablet to freeze the 
transitional image, as indicated in block 322. If the tip is pressed again, the frozen 
5 image is saved. The save options are available with or without the priority areas in 
effect. After the save has been accomplished, or if the user did not wish to save a 
transitional view of an image, the Compare option is complete and the routine 
branches to block 326. 

In an alternative embodiment of the invention, the Compare option allows a 

10 user to compare a modified image with any edits made to the modified image during 
the current editing session, i.e., before the changes are permanently saved to the 
modified image. Specifically, with reference to FIGURE 2, this embodiment of the 
Compare option contrasts the image in the current image buffer 54 with the image in 
the modified image buffer 52. As discussed above, this embodiment of the Compare 

15 option may also be used in conjunction with the Prioritize option to allow the user to 
select priority areas for comparison. In this case, the priority areas transition from the 
modified to the current image, while the modified image only is displayed in the other 
(nonselected areas) areas. 

At block 326, a test is made to determine whether the user wishes to view a 

20 split image. The Split Image option is used on a frontal picture only, and allows a 
patient to see his or her asymmetries. If a split image view is desired, the user is 
prompted to select an image, e.g., original or modified, at block 330. At block 332, a 
vertical centerline is displayed on top of the selected image. The user is then 
prompted to position the centerline at the location desired, as indicated at block 334. 

25 Typically, the centerline will be positioned to dissect the image into equal halves, 
using the nose and the eyes as reference points. At block 336, the aesthetic imaging 
system displays two images, one showing the left halves pieced together and the other 
the right halves pieced together. Specifically, the aesthetic imaging system will 
produce an inverse image of the left (right) half and then add it to the left (right) half. 

30 FIGURES 15A-15C illustrate the resultant images that are displayed when the 

Split Image option is invoked. In FIGURE 15 A, a frontal image 350 of a patient is 
shown, including a centerline 352 that has been positioned at the center of the patient 
by a user. FIGURE 15B is an illustration of the left halves of the image after being 
pieced together by the aesthetic imaging system, as indicated by reference 

35 numeral 354. FIGURE 15C is an illustration of the right halves of the image, as 
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To further reduce the distraction caused by the menu bar, preferably the menu 
bar 412 may be moved by a user to different locations on the screen 414. For 
example, the menu bar may be moved by a user to the bottom of the screen. 
Particularly when editing the face of a patient on the screen, the majority of the editing 
5 will take place on the upper two-thirds of the screen. Locating the menu bar on the 
bottom of the screen therefore positions the menu bar away from the area on which 
the patient should remain focused. It will be appreciated that techniques for moving a 
menu bar to various locations on a screen are known to those skilled in the art of 
designing user interfaces. 

10 FIGURE 21 is a flow chart of an exemplary routine for implementing a zoom 

feature in the aesthetic imaging program. The room feature allows a user to increase 
the scale of the image to better view a selected area and to improve the ability of the 
user to edit fine details in the image. At a block 400, a zoom point is selected by a 
user. The zoom point identifies the center of the image to be expanded under the 

15 control of the user. At a block 402, the picture is adjusted to position the zoom point 
at the center of the monitor. Centering the picture ensures that as the image is 
enlarged, the portion of the image surrounding the zoom point will be displayed. At a 
block 404, the user is allowed to input a desired magnification factor. Preferably, the 
magnification factor is selected by floating the pen from the bottom (minimum 

20 magnification) to the top (maximum magnification) of the tablet. As the user floats 
the pen over the tablet, the image on the monitor is correspondingly magnified and 
redisplayed at a block 406. With each redisplay of the image, at a block 408 a test is 
made to see if the user has frozen the image by pressing or tipping the pen. Once the 
image is frozen at a desired magnification, a user can manipulate the image using the 

25 array of drawing tools described above. It will be appreciated that for very fine work, 
such as removing small wrinkles surrounding a patient's eyes, the ability to magnify 
the image increases the quality of the editing that may be achieved. 

Several refinements of the zoom feature may be incorporated in the aesthetic 
imaging program to improve the results of the zoom. For example, a smoothing 

30 function may be incorporated in the zoom feature to ensure that as the image is 
magnified it does not become "pixelly" or grainy. The smoothing function may be 
implemented in software. Preferably, however, the smoothing function is 
implemented in hardware, such as a smoothing feature provided in the Targa 2000 
board described above and incorporated in the aesthetic imaging system. Further, 

35 feedback may be provided to the user in the form of a numerical display on the image 
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VI. Post Editing Image Manipulation 

The aesthetic imaging system 20 contains several modules to allow an image 
to be manipulated to improve the quality of the image, to analyze the image, or to 
prepare the image for meetings or presentations. FIGURES 22A-22D illustrate an 
5 exemplary routine for implementing some of the modules in the aesthetic imaging 
system. The modules are accessible from a pull-down menu or other function key. 

The first module that the user may select is a color correction module 500 that 
allows the user to adjust the color of an image so that the image matches the color of 
a target image. If the user has selected the color correction module, at a block 502 

10 the user is allowed to select an original image and a target image. FIGURE 23 depicts 
a preferred screen of the aesthetic imaging system when the color of an image is to be 
corrected. On the left hand side of the screen, a number of stored images 620a, 620b, 
620c, 620d, and 620e are displayed in an image index 620. From the image index, the 
user may select an original image to be manipulated and a target image from which the 

15 color is to be copied. The user selects the original and target images by moving the 
cursor over the desired image, selecting the image by tipping the pen on the tablet, 
and dragging the image to the appropriate location on the screen. The original image 
is selected by dragging an image from the image index 620 to an original image 
box 622, while the target image is selected by dragging an image to a target image 

20 box 624. When selected, each of the images is displayed in the appropriate box by the 
aesthetic imaging system. 

Returning to FIGURE 22A, at a block 504 the user is allowed to select 
comparison points on the original image and the target image. An original 
comparison point 626 is selected on the original image by moving the cursor to the 

25 appropriate point on the original image and tipping the pen on the tablet. Similarly, a 
target comparison point 628 is selected on the target image by appropriately moving 
the cursor and tipping the pen on the tablet. As discussed below, the comparison 
points define the color correction that is made to the original image. 

The image captured by the image capture board in the aesthetic imaging 

30 system is comprised of a number of pixels, each pixel having an 8-bit red component, 
an 8-bit green component, and an 8-bit blue component. At a block 506, a 
comparison is made of the red, green, and blue (RGB) components of the pixels in the 
regions immediately surrounding the selected comparison points. That is, the RGB 
components of the pixels in the region immediately surrounding the original 

35 comparison point are compared with the RGB components of the pixels in the region 
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color composition, and modify an original image to match the target image so that an 
accurate comparison may be made between the images. 

At a decision block 514, the user may continue to adjust the color of the image 
if the desired color is not achieved. At a block 504, the user may move the original 
5 comparison point 626 or the target comparison point 628 to change the image 
depicted in the adjusted image box 630. Such modifications occur in real time, 
allowing the user to quickly try a variety of different color corrections. If the desired 
color of the image is achieved, at a block 516 the adjusted image contained in the 
adjusted image box 630 is stored. It will be appreciated that the adjusted image may 

10 be stored over the old original image, or stored at a new memory location to preserve 
both the original image and the adjusted image. Once an offset is calculated, the user 
may also continue to modify a number of images using the same offset To continue 
to modify images, the user selects another image from the image index 620 and drags 
the image to the original image box 622. The selected original image is automatically 

15 modified using the current offset, and displayed in the adjusted image box 630. The 
adjusted image is then stored. In this manner, a user can quickly adjust the color of a 
number of images so that the images may be accurately compared. 

Returning to FIGURE 22A, the second module that the user may select is an 
orientation correction module 520 to allow the size and orientation of an image to be 

20 corrected. If the user has selected the orientation correction module, at a block 522 
the user is allowed to select an original image and a target image from an image index. 
As depicted in FIGURE 24, an original image is selected from the image index 620 
and displayed in an original image box 640. A target image is selected and displayed 
in a target image box 642. Oftentimes, the orientation or size of the patient's image in 

25 the original image will be different from the target image. For example, in the 
representative images shown in FIGURE 24, the patient's head in the original image is 
tilted slightly from the orientation in the target image. To correct for the patient tilt in 
the original image, at a block 524 the user selects at least three reference points 644 
on the target image. At a block 526, the user selects at least three corresponding 

30 reference points 648 on the original image. The reference points selected on the 
original image should be located at approximately the same location on the patient's 
image as they are located on the target image. 

At a block 528, a calculation is made to determine the transform necessary to 
rotate and size the original image so that it will be the same size and orientation as the 

35 target image. Those skilled in the art will appreciate that the relative size of the 
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underlying image. While each measurement option will be discussed as being 
implemented on a separate image below, it will be appreciated that the various 
different measurements may be combined on a single image. 

At a decision block 542, a test is made to determine if the user has selected the 
5 angle measurement option from the measurement module. FIGURE 25A depicts the 
screen of the aesthetic imaging system when the user selects the angle measurement 
option. At a block 544, the user initially selects an origin 650 for the angle 
measurement by moving a cursor 656 to the appropriate location on the screen and 
tipping the pen on the tablet. To make an angle measurement, at blocks 546 and 548 

10 the user must define a first line 658 and a second line 660 that intersect at the origin. 
To define the first line, a dashed line 652 is displayed that passes through the origin 
after the origin has been selected. A solid line segment 654 extends from the origin 
along the dashed line to a point where the cursor 656 is located. Movement of the 
cursor 656 along the dashed line 652 causes the length of the solid line segment 654 

15 to change. Movement of the cursor around the origin 650 causes the angle of the 
dashed line with respect to the origin to change. When a desired location of the 
dashed line is achieved, such as parallel with a patient's feature, the user tips the pen 
to set the location and length of the first line 658. After the first line is defined, the 
dashed line 652 extending through the origin is again displayed to allow the user to 

20 define the second line. Movement of the cursor changes the direction and length of 
the line segment along the dashed line. As depicted in FIGURE 25A, the second line 
660 is defined so that it forms an angle 662 with respect to the first line 658. The user 
selects the orientation and length of the second line by tipping the pen when the cursor 
is at the desired location. 

25 After the user has defined the two lines on the patient's image, at a block 550 

the angle between the two lines is calculated and displayed on the image. Two lines 
that intersect at the origin 650 will define two angles, one greater than or equal to 
180 degrees, and one less than or equal to 180 degrees. Preferably, the aesthetic 
imaging system automatically displays the calculated angle that is less than or equal to 

30 180 degrees. The angle is displayed at a location 664 that is opposite the defined 
angle. While preferably the angle is displayed in degrees, it will be appreciated that 
the angle may also be displayed in other units of measure, such as radians. 

In addition to displaying a single angle, the aesthetic imaging system also 
allows the user to link multiple angles together. For example, as shown in 

35 FIGURE 25A, three angles are defined by a common line 666. To allow the user to 
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by the user is calculated by known methods and displayed at a location 690 adjacent 
the defined area. Those skilled in the art will appreciate that the calculation of the 
area 690 is a simple matter once the image has been calibrated. The area may be 
expressed in English units, metric units, or any other units of measure used to 
5 calibrate the image. Further, the user may define any area within the image to be 
measured. 

Similar to making an area measurement, the user may also make a distance 
measurement on the image. At a decision block 568, a test is made to determine if the 
user has selected the distance measurement option. FIGURE 25C depicts an image in 

10 which a user has made several distance measurements. If the user has not calibrated 
the image, at a block 570 the image must first be calibrated in the manner described 
above. If the image is already calibrated, at a block 572 the user defines a line 
segment by selecting a first end point 700 and a second end point 702. After selecting 
the two end points, a line 704 is displayed on the image connecting the two aid 

15 points. At a block 574, the two-dimensional distance between the two points is 
calculated and displayed at a location 706 adjacent the line. Those skilled in the art 
will appreciate that calculating a two-dimensional distance on the Image is 
straightforward once the image has been calibrated. The distance may be displayed in 
English units, metric units, or other standard units of measure. 

20 As with the angle measurements, the aesthetic imaging system also allows a 

user to chain multiple distance measurements together. If the user has selected the 
chaining option at a decision block 576, the user is allowed to enter another end point 
after the first two end points are displayed at a block 578. The third and subsequent 
end points are connected to the preceding end point by a line, and the distance of each 

25 line displayed. As depicted in FIGURE 25C, the user may use the chaining option to 
quickly enter a number of lines 708 to measure around a desired area or between 
multiple points. 

With reference to FIGURE 22C, the fourth measurement that a user may 
make on an image is a proportion measurement. At a decision block 580, a test is 

30 made to determine if the user has selected the proportion measurement option. If the 
user has selected a proportion measurement, the user is presented the option of 
making a horizontal proportion measurement (comparing the horizontal relationship 
of features in the image) or a vertical proportion measurement (comparing the vertical 
relationship of features in the image). At a decision block 582 a test is made to 

35 determine if the horizontal or the vertical proportion measurement has been selected. 
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second end point 732 located at a distance from the eyebrow. After defining the line 
segment, at a block 604 the user enters a label corresponding to the line segment. As 
shown in FIGURE 26, a transparent box 734 is displayed to the user at the location 
where the label will be displayed. The user enters the label using a keyboard or other 
5 technique, such as selecting predefined labels from a pull-down menu. The box 734 
used to enter the label is preferably transparent to allow the user to see the image of 
the patient through the box when a label is being entered. 

At a decision block 606 the user is allowed to enter multiple labels, each of 
which are linked to one or more structures on the drawing. For example, a label 736 

10 may be linked to two or more One segments that point to different structures on the 
image. In FIGURE 26, the label "Lips" is connected by two line segments to each of 
the patient's lips. The labels are also linked to the line segments so that if the labels 
are moved, the line segments are correspondingly moved to continue to connect the 
label with the associated structure. For example, the label "Eye," which points to the 

15 eye of the image in FIGURE 26, may be selected by the user and dragged from a first 
location 738 to a second location 740 as indicated by the directional arrow. Moving 
the label "Eye" automatically moves the line segment associated with the label so that 
the label and the structure remain linked. 

The aesthetic imaging system modules disclosed above are particularly useful 

20 for allowing an imaging professional to manipulate an image to improve the 
presentation of the image to the patient or to colleagues. Such modules allow a more 
accurate comparison to be made between two or more images. The result is a more 
satisfying and beneficial imaging session for a patient. 

VII. Treatment Simulations 

25 The aesthetic imaging system of the present invention incorporates a module 

for simulating the results of laser resurfacing treatment. FIGURE 27A depicts a 
representative screen of the aesthetic imaging system when the laser resurfacing 
module has been selected. The laser resurfacing module allows the aesthetic imaging 
system to display an image of the patient that approximates how the patient will 

30 appear immediately following a laser resurfacing treatment and throughout the healing 
process.. 

The steps required to simulate the results of the laser resurfacing treatment are 
depicted in the flow chart of FIGURE 27B. At a block 754, the user initially selects 
an image to be edited, and defines a region in which the laser resurfacing treatment is 
35 to be simulated. The region is defined by the user by circling the desired area. For 
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As the mask becomes more opaque, more blurring of the image occurs. Preferably, 
the opacity of the alpha mask 764 is tied to the movement of the cursor to allow the 
user to change the blurring of the image by floating the pen from the top to the 
bottom of the tablet. 

5 Because of computational limitations, the blurred image 770 is preferably only 

calculated once and the amount of blurring varied by changing the opacity of the alpha 
mask 764 as described above. Alternatively, those skilled in the art will recognize that 
with additional processing power, the amount of blurring may be varied by directly 
changing the response of the low pass filter 766. Using the alternative technique, the 

10 low pass filter response may be tied to the movement of the cursor so that the user 
may directly vary the blurriness of the selected region by floating the pen from the top 
to the bottom of the tablet. 

When the user has achieved a desired amount of blurriness to simulate the 
representative treatment results within the region 752, the user freezes the image of 

15 the patient by tipping the pen on the tablet. An image 772 representative of the 
anticipated treatment results (after the patient has healed) is then stored in a buffer in 
the aesthetic imaging system. 

To simulate the redness and inflammation immediately following the 
resurfacing treatment, at a block 758 the user selects an amount of redness to be 

20 applied to the treated region. The amount of redness may be preset, or selected by the 
user from a scale of redness. FIGURE 27D diagrammatically represents the process 
for displaying the selected region 752 in a reddened state. The alpha mask 764 is 
added to a solid image 774 containing pixels of the desired redness, and the alpha 
channel values in the resulting mask changed to make the mask semi-transparent. The 

25 mask and colored image is then superimposed over the image 772 of the 
representative treatment results after healing. The result is an image 776 that is 
representative of the image of the patient after the treatment has been performed, but 
before the patient has healed. 

Returning to FIGURE 27B, the user is allowed to fade the redness of the 

30 image at a block 760 to show how the patient will likely look during the healing 
process. In order to fade the redness of the image, the alpha values in the alpha 
mask 764 are reduced until the entire mask is transparent. As the mask becomes more 
transparent, more of the underlying image is visible to the user. Preferably, the 
transparency of the alpha mask 764 is tied to the movement of the cursor to allow the 

3 5 user to change the redness of the image by floating the pen from the top to the bottom 
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The embodiments of the invention in which an exclusive property or privilege 
is claimed are defined as follows: 

1. In an aesthetic imaging system of the type that manipulates an image of 
a patient to simulate the effects of a medical procedure, a method for simulating the 
results of a laser resurfacing treatment, comprising the steps of: 

obtaining an original image of the patient; 
5 defining an area in the image in which the laser resurfacing is to be simulated; 

creating a mask whereby the areas outside the region in which the laser 
resurfacing is not to be simulated is transparent and the areas in which the laser 
resurfacing is to be simulated is opaque; 

creating a blurred copy of the original image in the area in which the laser 
1 0 resurfacing is to be simulated; and 

adding the mask to the blurred copy of the original image and superimposing 
the sum of the blurred image with the mask over the original image to simulate the 
laser resurfacing. 

2. The method of Claim 1, further comprising a method for connecting 
the color of an image comprising the steps of: 

selecting one or more comparison points in an original image and a target 

15 image; 

calculating an average red, green, and blue components of the pixels near the 
comparison points in the original image and the target image; 

determining a difference in the average red, green, and blue components of the 
pixels near the comparison points in the original and target images; and 
20 adjusting the red, green, and blue components of the pixels near the target 

points in the original image to equal the red, green, and blue components of the pixels 
in the target image. 
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AMENDED CLAIMS 

[received by the International Bureau on 30 April 1998 (30.04.98); 
original claims 1-5 replaced by new claims 1-13 (3 pages)] 

S 1. In an aesthetic imaging system allowing the manipulation of digital 

images comprised of a plurality of pixels, a method of displaying an edited digital 
image corresponding to an original digital image wherein the edited digital image 
differs from the original digital image by at least one edit made to the original digital 
image, the aesthetic imaging system including a processor, a memory, a monitor, and 
10 means for controlling a cursor displayed on the monitor, the method comprising: 

defining a plurality of regions in said edited digital image, each of the plurality 
of regions containing at least one edit that distinguishes each of the plurality of 
regions from a corresponding region in said original digital image; 

allowing a user to select from the plurality of regions a subset of regions; and 
IS displaying a prioritized digital image containing only the edits of the original 

digital image that are present in the subset of regions. 

2. The method of Claim 1 wherein the plurality of regions are defined by 
encircling the region on the edited digital image with the means for controlling the 
cursor displayed on the monitor. 

20 3. The method of Claim 1 wherein the plurality of regions are 

automatically defined by the aesthetic imaging system. 

4. The method of Claim 1 further comprising the step of assigning a tag 
corresponding to each of the plurality of regions. 

5. The method of Claim 4 wherein the tag is a textual identifier. 

25 6. The method of Claim 5, wherein the subset of regions are selected 

from a list of tags corresponding to each region. 

7. The method of Claim 1 wherein the original image and the prioritized 
image are displayed side-by-side. 

8. The method of Claim 1 wherein the prioritized image is superimposed 
30 over the original image, an opacity of the prioritized image being selectable by a user 

to allow the user to view a composite image of the prioritized image and the original 
image, the Composite image thereby displaying a variable amount of the edits 
performed on the original image. 
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5 calculating an offset value by subtracting an average of the RGB components 

of the pixels in a region surrounding the comparison point in the original image from 
an average of the RGB components of the pixels in a region surrounding the 
comparison point in the target image, 

adding the offset value to the RGB components of each pixel in the original 
10 image to correct the color of the original digital image; and 
displaying the color corrected original digital image. 

13. In an aesthetic imaging system allowing the manipulation of digital 
images comprised of a plurality of pixels, a method of correcting the size and 
orientation of an original digital image to substantially match a target digital image 
IS comprising: 

selecting an original digital image and a target digital image from a memory of 
the aesthetic imaging systems; 

selecting at least three reference points on the original digital image and at 
least three corresponding reference points on the target digital image; 
20 calculating a degree of rotation and a size adjustment based on the three 

reference points in the original and target digital images; 

rotating the original digital image based on the calculated degree of rotation 
and scaling the original digital image based on the calculated size adjustment to create 
a corrected digital image; and 
25 displaying the corrected digital image. 
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